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Abstract 

This paper presents the XTAG system, a grammar de- 
velopment tool based on the Tree Adjoining Grammar 
(TAG) formalism that includes a wide-coverage syn- 
tactic grammar for English. The various components 
of the system are discussed and preliminary evalua- 
tion results from the parsing of various corpora are 
given. Results from the comparison of XTAG against 
the IBM statistical parser and the Alvey Natural Lan- 
guage Tool parser are also given. 

1 INTRODUCTION 

XTAG is a large on-going project to develop a wide- 
coverage grammar for English, based on the Lexi- 
calized Tree Adjoining Grrammar (LTAG) formalism. 
LTAG is a lexicalized mildly-context sensitive tree 
rewriting system [Joshi et al, 1975; Schabes, 1990] 
that is closely related to Dependency Grammars and 
Categorial Grammars. Elementary trees in LTAG 
provide a larger domain of locality over which syn- 
tactic and semantic (predicate-argument) constraints 
are specified. XTAG also serves as an LTAG gram- 
mar development system consisting of a predictive 
left-to-right parser, an X-window interface, a mor- 
phological analyzer, and a part-of-speech tagger (also 
referred to as simply 'tagger'). 

2 SYSTEM DESCRIPTION 

Figure 1 shows the overall flow of the system when 
parsing a sentence. The input sentence is submit- 
ted to the Morphological Analyzer and the Tagger. 
The morphological analyzer retrieves the morpho- 
logical information for each individual word from 
the morphological database. This output is filtered 
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Figure 1: Overview of XTAG system 



in the P.O.S Blender using the output of the trigram 
tagger to reduce the part-of-speech ambiguity of the 
words. The sentence, now annotated with part-of- 
speech tags and morphological information for each 
word, is input to the Parser, which consults the syn- 
tactic database and tree database to retrieve the appro- 
priate tree structures for each lexical item. A variety 
of heuristics are used to reduce the number of trees 
selected. The parser then composes the structures to 
obtain the parse(s) of the sentence. 

2.1 Morphological Analyzer 

The morphology database [Karp et al., 1992] was 
originally extracted from 1979 edition of the Collins 
English Dictionary and Oxford Advanced Learner's 



Dictionary of Current English, and then cleaned up 
and augmented by hand. It consists of approximately 

317,000 inflected items, along with their root forms 
and inflectional information (such as case, num- 
ber, tense). Thirteen parts of speech are differen- 
tiated: Noun, Proper Noun, Pronoun, Verb, Verb 
Particle, Adverb, Adjective, Preposition, Comple- 
mentizer, Determiner, Conjunction, Interjection, and 
NounA'erb Contraction. Nouns and Verbs are the 
largest categories, with approximately 213,000 and 
46,500 inflected forms, respectively. The access time 
for a given inflected entry is 0.6 msec. 

2.2 Part-of-Speech Tagger 

A trigram part-of-speech tagger [Church, 1988], 
trained on the Wall Street Journal Corpus, is incor- 
porated in XTAG. The trigram tagger has been ex- 
tended to output the N-best parts-of-speech sequences 
[Soong and Huang, 1990]. XTAG uses this infor- 
mation to reduce the number of specious parses by 
filtering the possible parts-of-speech provided by the 
morphological analyzer for each word. The tagger 
decreases the time to parse a sentence by an average 
of 93%. 

2.3 Parser 

The system uses an Earley-style parser which has 
been extended to handle feature structures associated 
with trees [Schabes, 1990]. The parser uses a general 
two-pass parsing strategy for 'lexicalized' grammars 
[Schabes, 1988]. In the tree-selection pass, the parser 
uses the syntactic database entry for each lexical item 
in the sentence to select a set of elementary struc- 
tures from the tree database. The tree-grafting pass 
composes the selected trees using substitution and ad- 
junction operations to obtain the parse of the sentence. 
The output of the parser for the sentence / had a map 
yesterday is illustrated in Figure 2. The parse tree^ 
represents the surface constituent structure, while the 
derivation tree represents the derivation history of 
the parse. The nodes of the derivation tree are the tree 
names anchored by the lexical items. The composi- 
tion operation is indicated by the nature of the arcs; a 
dashed line is used for substitution and a bold line for 
adjunction. The number beside each tree name is the 
address of the node at which the operation took place. 
The derivation tree can also be interpreted as a de- 
pendency graph with unlabeled arcs between words 
of the sentence. 

Heuristics that take advantage of LTAGs have been 
implemented to improve the performance of the 

'Each node of the parse tree has a feature structure, not shown 
here, associated with it. 
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parser For instance, the span of the tree and the 
position of the anchor in the tree are used to weed out 
unsuitable trees in the first pass of the parser. Statisti- 
cal information about the usage frequency of the trees 
has been acquired by parsing corpora. This informa- 
tion has been compiled into a statistical database (the 
Lex Prob DB) that is used by the parser. These 
methods speed the runtime by approximately 87%. 

2.3.1 Heuristics for Ranking the Parses 

The parser generates the parses in a rank order This 
ranking is determined using a combination of heuris- 
tics, which are expressed as structural preferences for 
derivation, e.g. attachment sites of adjuncts, right- vs. 
left- branching structures, topicalized sentences, etc. 
Similar heuristics have been used for other parsers. 
See recent work by [Hobbs and Bear, 1994], [Mc- 
Cord, 1993],and [Nagao, 1994]. 

A partial list of the heuristics used in XTAG fol- 
lows: 

1. Prefer argument positions to adjunct positions 
(here, this amounts to preferring fewer adjunc- 
tion operations). 



2. For PPs other than of, attach to nearest site that 
is not a proper noun. 

3 . Prefer right-branching structure for sequences of 
adjectives, adverbs, and PPs. 

4. Prefer left-branching structure for sequences of 
nouns. 

5. Prefer high attachment (wide-scope) for a modi- 
fier and a sequence of modifiees of the same type 
(i.e. a PP following or preceding a coordinate 
NP, an adjective or determiner preceding a co- 
ordinate NP or sequence of Ns, an N preceding 
coordinate Ns). 

These rankings are used to control the number of 
sentences passed on to further levels of processing. 
In applications emphasizing speed, only the high- 
est ranked parse will be considered. In applications 
emphasizing accuracy, the top N parses can be con- 
sidered. 

2.3.2 Syntactic Database 

The syntactic database associates lexical items with 
the appropriate trees and tree families based on se- 
lectional information. The syntactic database entries 
were originally extracted from the Oxford Advanced 
Learner's Dictionary and Oxford Dictionary for Con- 
temporary Idiomatic English, and then modified and 
augmented by hand. There are more than 37,000 
syntactic database entries. Selected entries from this 
database are shown in Table 1 . Each syntactic entry 
consists of an INDEX field, the uninflected form under 
which the entry is compiled in the database, an ENTRY 
field, which contains all of the lexical items that will 
anchor the associated tree(s), a POS field, which gives 
the part-of-speech for the lexical item(s) in the ENTRY 
field, and then either (but not both) a TREES or FAM 
field. The trees field indicates a list of individual 
trees to be associated with the entry, while the FAM 
field indicates a list of tree families. A tree family, 
which corresponds to a subcategorization frame (see 
section 2.3.3), may contain a number of trees. A 
syntactic entry may also contain a list of feature tem- 
plates (fs) which expand out to feature equations to 
be placed in the specified tree(s). Any number of ex 
fields may be provided for example sentences. Note 
that lexical items may have more than one entry and 
may select the same tree more than once, using dif- 
ferent features to capture lexical idiosyncrasies (e.g. 
have). 



INDEX: have/26 

ENTRY: have 

POS: V 

TREES: /3Vvx 

FS: #VPr_ind, #VPr_past, #VPr_perfect-l- 

#VP_ppart, #VP_pass- 

EX: he had died; we had died 

INDEX: have/50 

ENTRY: have 

POS: V 

TREES: /3Vvx 

FS: #VP_inf 

EX: John has to go to the store. 

INDEX: have/69 

ENTRY: NPO have NPl 

POS: NPO V NPl 

FAM: TnxOVnxl 

FS: #TRANS-i- 

EX: John has a problem. 

INDEX: map/1 

ENTRY: NPO map out NPl 

POS: NPO V PL NPl 

FAM: TnxOVplnxl 

INDEX: map/3 

ENTRY: map 

POS: N 

TREES: ciN, ciNXdxN, /3Nn 

FS: #N_wh-, #N_refl- 

INDEX: map/4 
ENTRY: map 
POS: N 
TREES: aNXN 

FS: #N_wh-, #N_refl-, #N_plur 

Table 1 : Selected Syntactic Database Entries 

2.3.3 Tree Database 

Trees in the English LTAG framework fall into two 

conceptual classes. The smaller class consists of indi- 
vidual trees such as trees (a), (d), and (e) in Figure 3. 
These trees are generally anchored by non-verbal lex- 
ical items. The larger class consists of trees that are 
grouped into tree families. These tree families rep- 
resent subcategorization frames; the trees in a tree 
family would be related to each other transforma- 
tionally in a movement-based approach. Trees 3(b) 
and 3(c) are members of two distinct tree families. 
As illustrated by trees 3(d) and 3(e), each node of a 
tree is annotated with a set of features whose values 
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agr : <1> 
tense : <2> 
mode : <3> 
assign-case : <4> 
perfect : <5> 




agr 

tense : <2> [ ] 
mode : <3> [ ] 
assign-case : <4> [ ] 



(d) (e) 
Figure 3: Sample trees from the Tree Database 



perfect : <S> [ ]] 

na[] 



may be specified within the tree or may be derived 
from the syntactic database. There are 385 trees that 
compose 40 tree families, along with 62 individually 
selected trees in the tree database. 

2.3.4 Statistics Database 

The statistics database contains tree unigram frequen- 
cies which have been collected by parsing the Wall 
Street Journal, the IBM manual, and the ATIS corpus 
using the XTAG English grammar. The parser, aug- 
mented with the statistics database [Joshi and Srini- 
vas, 1994], assigns each word of the input sentence 
the top three most frequently used trees given the 
part-of-speech of the word. On failure the parser 
retries using all the trees suggested by the syntactic 
database for each word. The augmented parser has 
been observed to have a success rate of 50% without 
retries. 

2.4 X-Interface 

XTAG provides a graphical interface for manipulat- 
ing TAGs. The interface offers the following: 



• Menu-based facility for creating and modifying 
tree files and loading grammar files. 

• User controlled parser parameters, including the 

parsing of categories (S, embedded S, NP, DetP), 
and the use of the tagger (on/off/retry on failure). 

• Storage/retrieval facilities for elementary and 
parsed trees as text files. 

• The production of postscript files corresponding 

to elementary and parsed trees. 

• Graphical displays of tree and feature data struc- 
tures, including a scroll 'web' for large tree 
structures. 

• Mouse-based tree editor for creating and modi- 
fying trees and feature structures. 

• Hand combination of trees by adjunction or sub- 
stitution for use in diagnosing grammar prob- 
lems. 

Figure 4 shows the X window interface after a 
number of sentences have been parsed. 
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clerivecl-tree-889 
Bill wanted to go to Washington 

derived-tree-140368 

derived- tree- 140369 

derived- tree- 140370 
I couldn't understand him 

derived- tree- 140449 
Go to your room 

derived- tree- 140620 

derived- tree- 140621 

derived- tree- 140622 
he would not revesd which magazines he was considering 

derived- tree- 140763 

derived- tree- 140764 

derived- tree- 140765 

derived- tree- 140766 



parsers in the same class as XTAG. Although XTAG 
is being extended to handle sentence fragments, they 
are not included at present, and are thereby not re- 
flected in the data in Table 2. Statistical information 
from the parsed corpora described in Section 2.3.4 is 
presently used only for speeding the parser but not to 
tune the grammar to parse any specific corpus. Note 
then, that the data below does not involve any corpus 
training. 
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WSJ 


6364 


39.09% 


7.53 


IBM Manual 


1611 


75.42% 


6.14 


ATIS 


649 


74.42% 


6.0 



Table 2: Performance of XTAG on various corpora 



Figure 4: XTAG Interface 

3 ENGLISH GRAMMAR 

The morphology, syntactic, and tree databases to- 
gether comprise the English grammar. Lexical items 
not in the databases are handled by default mecha- 
nisms. The range of syntactic phenomena that can 
be handled is large and includes auxiliaries (includ- 
ing inversion), copula, raising and small clause con- 
structions, topicalization, relative clauses, infinitives, 
gerunds, passives, adjuncts, it-clefts, wh-clefts, PRO 
constructions, noun-noun modifications, extraposi- 
tion, determiner phrases, genitives, negation, noun- 
verb contractions and imperatives. Analyses for sen- 
tential adjuncts and time NP adverbials are currently 
being implemented. The combination of large scale 
lexicons and wide phenomena coverage result in a 
robust system. 

4 CORPUS PARSING AND EVALU- 
ATION 

XTAG has recently been used to parse the Wall Street 
Journal^, IBM manual, and ATIS corpora as a means 
of evaluating the coverage and correctness of XTAG 
parses. For this evaluation, a sentence is consid- 
ered to have parsed correctly if XTAG produces 
parse trees. Verifying the presence of the correct 
parse among the parses generated is done manually 
at present. Table 2 shows the preliminary results. We 
will present more complete and rigorous results by the 
time of the conference and compare them with other 

^Sentences of length <= 15 words 



4.1 Comparison with IBM Parser 

A more detailed experiment to measure the crossing 
bracket accuracy of the XTAG-parsed IBM-manual 
sentences has been performed. Of the 1600 IBM 
sentences that have been parsed (those available from 
the Penn Treebank [Marcus et al, 1993]), only 67 
overlapped with the IBM-manual treebank that was 
bracketed by University of Lancaster.^ The XTAG- 
parses for these 67 sentences were compared^ with 
the Lancaster IBM-manual treebank. 

Table 3 shows the results obtained in this experi- 
ment. It also shows the crossing bracket accuracy of 
the latest IBM statistical parser [Jelinek et al, 1994] 
on the same genre of sentences. Recall is a measure 
of the number of bracketed constituents the system 
got right divided by the number of constituents in 
the corresponding Treebank sentences. Precision is 
the number of bracketed constituents the system got 
right divided by the number of bracketed constituents 
in the system's parse. 

Based on the present data, we beheve our results 
will be consistent for the complete XTAG-parsed 
IBM corpus; we plan to evaluate the XTAG parses 
against the Penn Treebank. In addition, we are pars- 
ing the Lancaster sentences, and adding those to the 
XTAG IBM corpus. 

While the crossing-brackets measure is useful for 
comparing the output of different parsers, we believe 
that it is a somewhat inadequate method for evaluat- 
ing a parser like XTAG for two main reasons. First, 

'The treebank was obtained through Sahm Roukos 
(roukos@watson.ibm.com) at IBM. 

''We used the parseval program written by Phil Harison 
(phil@atc.boemg.com). 
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Table 3: Performance of XTAG on IBM-manual sen- 
tences 

the parse generated by the XTAG system is much 
richer in its representation of the internal structure of 
certain phrases than those present in manually cre- 
ated treebanks. Even though the Lancaster treebank 
is more detailed in terms of bracketing than the Penn 
Treebank, it is not complete in its bracketing of the 
internal structure of noun phrases. As a result of com- 
paring the XTAG parse with a skeletal representation, 
the precision score is misleadingly low for the XTAG 
system. 

A second reason that the crossing bracket measure 
is inadequate for evaluating XTAG is that the pri- 
mary structure in XTAG is the derivation tree from 
which the bracketed tree is derived. Two identical 
bracketings for a sentence can have completely dif- 
ferent derivation trees. A more direct measure of the 
performance of the XTAG parser would evaluate the 
derivation structure, which captures the dependencies 
between words. 

4.2 Comparison with Alvey 

We also compared the XTAG parser to the Alvey 
Natural Language Tools (ANLT) Parser, and found 
that the two performed comparably. We parsed the set 
of LDOCE Noun Phrases presented in Appendix B of 
the technical report [Carroll, 1993], using the XTAG 
parser. The technical report presents the ranking of 
the correct parse and also gives the total number of 
derivations for each noun phrase. In this experiment, 
we have compared the total number of derivations 
obtained from XTAG with that obtained from the 
ANLT parser. 

Table 4 summarizes the results of this experiment. 
A total of 143 noun phrases were parsed. The NPs 
which did not have a correct parse in the top three 
derivations for the ANLT parser were considered as 
failures for ANLT. The maximum and average num- 
ber of derivations columns show the highest and the 
average number of derivations produced for the NPs 
that have a correct derivation in the top three deriva- 
tions. For the XTAG system, performance results 
with and without the POS tagger are shown.^ 



Table 4: Comparison of XTAG and ANLT Parser 

It would be interesting to see if the two systems 
performed similarly on a wider range of data. In 
[Carroll, 1993], only the LDOCE NPs are annotated 
with the number of derivations; we are interested in 
getting more data annotated with this information, in 
order to make further comparisons. 

5 COMPUTER PLATFORM 

XTAG was developed on the Sun SPARC station se- 
ries, and has been tested on the Sun 4 and HP BOB- 
CATs series 9000. It is available through anonymous 
ftp, and requires 20MB of space. Please send mail to 
xtag-request@Iinc.cis.upenn.edu for ftp instructions 
or more information. XTAG requires the following 
software to run: 

• A machine running UNIX and XI 1R4. Previous 
releases of X will not work. X11R4 is free 
software available from MIT. 

• A Common Lisp compiler which supports the 
latest definition of Common Lisp (Steele's Com- 
mon Lisp, second edition). XTAG has been 
tested with Lucid Common Lisp 4.0 and Alle- 
gro 4.0.1. 

• CLX version 4 or higher. CLX is the lisp equiv- 
alent to the Xlib package written in C. 

• Mark Kantrowitz's Lisp Utilities from CMU: 
logical-pathnames and defsystem. 

The latest version of CLX (R5.0) and the CMU 
Lisp Utilities are provided in our ftp directory for 
your convenience. However, we ask that you refer to 
the appropriate source for updates. 

The morphology database component [Karp et al., 
1992], no longer under licensing restrictions, is avail- 
able as a separate system from the XTAG system. Ftp 
instructions and more information can be obtained by 
mailing requests to Iex-request@Iinc.cis.upenn.edu. 



^Because the NPs are, on average, shorter than the sentences 
on which it was trained, the performance of the POS tagger is 



significantly degraded. For this reason, the table contains infor- 
mation about the performance of XTAG both with and without 
the tagger. 
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